Skip to content

Add testcontainer-based JDBC tests#84

Draft
ostinru wants to merge 11 commits intoapache:mainfrom
ostinru:jdbc-testcontainers
Draft

Add testcontainer-based JDBC tests#84
ostinru wants to merge 11 commits intoapache:mainfrom
ostinru:jdbc-testcontainers

Conversation

@ostinru
Copy link
Copy Markdown
Collaborator

@ostinru ostinru commented Mar 20, 2026

Add testcontainers-bassed automation tests

For new tests I used testcontainers. This is a shift in the way we running automation tests:

  • new tests run on host machine (you can run tests from your IDE)
    +----------------------------------------------------------------+
    |                              Host                              |
    |  +-------------------------------+  +-----------------------+  |
    |  |           Docker              |  |        Docker         |  |
    |  |   [Cloudberry] --> [PXF] ------------> [Database]        |  |
    |  |                               |  |                       |  |
    |  +-------------------------------+  +-----------------------+  |
    |                                                                |
    |                         [Automation]                           |
    +----------------------------------------------------------------+
    
  • new test framework manages Docker containers on its own:
    • it will start only required containers
    • all test-specific code is located near test (not in a single bash-file)
    • it opens a way to run highly customised containers (kerberos tests?)
  • No jSystemCore. This framework have not active development for couple years. New classes uses same TestObject pattern, but have no direct dependency on jSystemCore.

Containers

New framework uses shrinked version of pxf-cbdb-dev:

  • no hadoop ecosystem in the same container
  • no more file permission issues - we copy files to container instead of mapping (binding) them in RW mode
  • faster cloudberry + PXF builds:
    • no golang & pxf-cli - it is unused
    • gradle is baked into docker image - no downloads during container run
    • map system-wide gradle cache to container as RO cache

Add jdbc-pxf-drivers project

Add new jdbc-pxf-drivers project to server. It excluded from default DEB-package build. Explicit actions required to install jdbc-drivers.

@ostinru
Copy link
Copy Markdown
Collaborator Author

ostinru commented Mar 20, 2026

Early publishing PR draft. It is not finished. However it shows main idea.

@ostinru ostinru force-pushed the jdbc-testcontainers branch 2 times, most recently from 9e4a6f8 to 2521cea Compare March 24, 2026 15:09
ostinru and others added 9 commits March 25, 2026 17:30
…issue

```
java.lang.IllegalAccessError: tried to access method org.apache.commons.io.build.AbstractStreamBuilder.getInputStream()Ljava/io/InputStream; from class org.apache.commons.compress.compressors.xz.XZCompressorInputStream
	at org.apache.commons.compress.compressors.xz.XZCompressorInputStream.<init>(XZCompressorInputStream.java:150)
	at org.apache.commons.compress.compressors.xz.XZCompressorInputStream.<init>(XZCompressorInputStream.java:166)
	at org.apache.avro.file.XZCodec.decompress(XZCodec.java:74)
	at org.apache.avro.file.DataFileStream$DataBlock.decompressUsing(DataFileStream.java:392)
	at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:226)
	at org.apache.avro.tool.DataFileReadTool.run(DataFileReadTool.java:97)
	at org.apache.cloudberry.pxf.automation.components.hdfs.Hdfs.writeJsonFileFromAvro(Hdfs.java:487)
	at org.apache.cloudberry.pxf.automation.features.avro.HdfsWritableAvroTest.fetchAndVerifyAvroHcfsFiles(HdfsWritableAvroTest.java:448)
	at org.apache.cloudberry.pxf.automation.features.avro.HdfsWritableAvroTest.generateSchemaPrimitive(HdfsWritableAvroTest.java:146)
```
@ostinru ostinru force-pushed the jdbc-testcontainers branch from 2521cea to 1a55c92 Compare March 25, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant